Semantic Fuzzing with Zest

Programs expecting structured inputs often consist of both a syntactic analysis stage, which parses raw input, and a semantic analysis stage, which conducts checks on the parsed input and executes the core logic of the program. Generator-based testing tools in the lineage of QuickCheck are a promising way to generate random syntactically valid test inputs for these programs. We present Zest, a technique which automatically guides QuickCheck-like randominput generators to better explore the semantic analysis stage of test programs. Zest converts random-input generators into deterministic parametric generators. We present the key insight that mutations in the untyped parameter domain map to structural mutations in the input domain. Zest leverages program feedback in the form of code coverage and input validity to perform feedback-directed parameter search. We evaluate Zest against AFL and QuickCheck on five Java programs: Maven, Ant, BCEL, Closure, and Rhino. Zest covers 1.03x-2.81x as many branches within the benchmarks semantic analysis stages as baseline techniques. Further, we find 10 new bugs in the semantic analysis stages of these benchmarks. Zest is the most effective technique in finding these bugs reliably and quickly, requiring at most 10 minutes on average to find each bug.Comment: To appear in Proceedings of 28th ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA'19

Declarative Experimentation in Information Retrieval Using PyTerrier

The advent of deep machine learning platforms such as Tensorflow and Pytorch, developed in expressive high-level languages such as Python, have allowed more expressive representations of deep neural network architectures. We argue that such a powerful formalism is missing in information retrieval (IR), and propose a framework called PyTerrier that allows advanced retrieval pipelines to be expressed, and evaluated, in a declarative manner close to their conceptual design. Like the aforementioned frameworks that compile deep learning experiments into primitive GPU operations, our framework targets IR platforms as backends in order to execute and evaluate retrieval pipelines. Further, we can automatically optimise the retrieval pipelines to increase their efficiency to suite a particular IR platform backend. Our experiments, conducted on TREC Robust and ClueWeb09 test collections, demonstrate the efficiency benefits of these optimisations for retrieval pipelines involving both the Anserini and Terrier IR platforms

Development of a Data Fusion Framework to support the Analysis of Aviation Big Data

The Federal Aviation Administration (FAA) is primarily responsible for the advancement, safety, and regulation of civil aviation, as well as overseeing the development of the air traffic control system in the United States. As such, it is faced with tremendous amounts of data on a daily basis. This data, which comes in high volumes, in various formats, from disparate sources and at various frequencies, is used by FAA analysts and researchers to make accurate forecasts, improve the safety and operational performance of their operations, and streamline processes. However, by its very nature, aviation Big Data presents a number of challenges to analysts: it impedes their ability to get a real-time picture of the state of the system, identify trends and operational patterns, make real-time predictions, etc. As such, the overarching objective of the present effort is to support FAA through the development of a data fusion framework to support the analysis of aviation Big Data. For the purpose of this research, three datasets were considered: System-Wide Information Management (SWIM) Flight Publication Data Service (SFDPS), Traffic Flow Management System (TFMS), and Meteorological Terminal Aviation Routine (METAR). The equivalent of one day of data was retrieved from each dataset, parsed and fused. A use case was then used to illustrate how a data fusion framework could be used by FAA analysts and researchers. The use case focused on predicting the occurrence of weather-related Ground Delay Programs (GDP) at the Newark (EWR), La Guardia (LGA), and Boston Logan (BOS) International Airports. This involved developing a prediction model using the Decision Tree Machine Learning technique. Evaluation metrics such as Matthew’s Correlation Coefficient were then used to evaluate the model’s performance. It is expected that a data fusion framework, once integrated within the FAA’s Computing and Analytics Shared Services Integrated Environment (CASSIE) could be used by analysts and researchers alike to identify trends and patterns and develop efficient methods to ensure that the U.S. civil and general aviation remains the safest in the world